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1.  Introduction 


The  purpose  of  this  report  is  to  review  the  eurrent  evidenee  relating  to  trust  in  human-automation 
interaetion  and  to  eompare  these  findings  with  our  prior  researeh  on  the  more  speeifie  area  of 
trust  in  human-robot  interaction  (HRI)  (Hancock  et  ah,  2011).  Since  automation  has  been  shown 
to  be  a  key  dimension  of  a  robot  (including  unmanned  systems),  identifying  the  factors  that  can 
impact  trust  between  humans  and  automation  may  significantly  increase  our  understanding  of 
trust  and  its  applications  to  HRI. 

Previous  meta-analytic  findings  on  human-robot  trust  helped  to  formulate  and  formalize  a  triadic 
model  of  trust,  which  categorized  the  factors  influencing  trust  development  as  human-related, 
robot-related,  or  environment-related  characteristics  (figure  1)  (Hancock  et  ah,  2011).  Hancock 
and  his  colleagues  (2011)  established  the  great  importance  of  robot  performance  consistency 
(e.g.,  reliability)  and  allied  attributes  in  the  development  of  trust  in  HRI.  Environmental 
characteristics  (e.g.,  team  collaboration,  task  type)  were  found  to  influence  trust  in  HRI  but  only 
to  a  lesser  degree  than  the  reliability  factor.  Further,  this  work  revealed  that  human-related 
factors  did  not  appear  to  play  a  large  role  in  the  development  of  trust;  however,  limited  empirical 
research  was  available  regarding  human-related  dimensions  and  the  influences  on  trust  in  robots 
at  the  present  time.  The  resulting  model  of  HRI  trust  must  thus  be  interpreted  with  caution,  as  the 
descriptive  model  may  change  as  the  dearth  of  current  experimentation  was  addressed. 
Additionally,  the  research  used  to  construct  the  trust  model  was  limited  only  to  empirical  studies 
documenting  human  interaction  with  embodied  robots,  which  is  a  specific  subfield  of 
automation.  Thus,  our  prior  meta-analysis  did  not  include  an  assessment  of  all  other  types  of 
automation  (e.g.,  decision  support  systems).  Consequently,  an  examination  of  the  trust  in  the 
larger  body  of  automation  literature  may  increase  our  understanding  of  human-robot  trust.  The 
following  represents  this  effort  using  a  quantified  meta-analytic  technique. 
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Figure  1.  Human-Robot  trust  (Hancock  et  al.,  2011). 

1.1  Foundation 

Technical  developments  have  made  it  possible  to  automate  tasks  that  have  traditionally  been 
carried  out  by  humans.  Automated  systems  can  be  used  in  environments  that  are  high  risk  for 
humans  (e.g.,  hazardous,  life-threatening  environments),  in  tasks  that  humans  do  not  want  to 
perform  (e.g.,  boring  and  repetitious  tasks),  and  in  tasks  where  a  machine  is  more  accurate  or 
reliable  than  a  human  (e.g.,  mathematical  computations,  robotics  manufacturing,  and  autopilot 
activity).  Parasuraman  and  colleagues  (2000)  have  provided  a  review  of  system  functions  that 
can  and  should  be  automated  based  on  their  cost  of  action  consequences  on  human  performance. 
As  such,  automation  is  now  available  in  multiple  domains,  from  the  home  (e.g.,  refrigerator)  to 
the  local  environment  (e.g.,  automated  teller  machine)  to  the  battlefield  (e.g.,  unmanned 
vehicles).  For  example,  within  robotics,  unmanned  aerial  systems  are  designed  to  be  used  in 
situations  where  machines  provided  better  sustained  performance  than  a  human,  when  there  are 
differing  political  and  human  costs  in  high-risk  environments,  and  when  there  is  an  increased 
likelihood  for  mission  success  (Office  of  the  Secretary  of  Defense,  2005).  Further,  in  the  final 
Task  Force  Report  on  the  role  of  autonomy  in  Department  of  Defense  systems,  the  Defense 
Science  Board  (DBS,  2012)  suggested  that  unmanned  systems  are  making  a  significant  impact 
on  warfare  worldwide.  This  report  stated  that  the  value  of  unmanned  systems  is  not  the 
replacement  of  a  human  but  human-system  collaboration  (i.e.,  the  capability  to  extend  and 
complement  human  capability  without  degradation  because  of  fatigue  or  lack  of  attention); 
however,  the  major  challenge  is  the  lack  of  trust  that  the  system  will  perform  as  intended. 
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In  addition,  automation  literature  has  explored  the  impaets  of  trust  on  performanee  outeomes  and 
trust  ealibration  strategies  in  greater  depth  than  traditional  human-robot  trust  literature.  For 
example,  trust  in  automation  is  directly  tied  to  the  outcome  measurements  of  human-automation 
interaction,  specifically  automation  use  (see  Beck  et  ah,  2007;  Moray  et  ah,  2000;  Muir  and 
Moray,  1996;  Parasuraman  et  ah,  1993).  These  include  human  reliance  on  automation  and 
human  compliance  with  automation  (see  also  Barnett,  2007;  Chen  and  Terrence,  2009;  Dzindolet 
et  ah,  2003;  Lee  and  Moray,  1992;  Lee  and  Moray,  1994;  Lee  and  See,  2004).  In  addition,  trust 
development  is  related  to  performance  outcomes  that  lead  to  safety  decisions  (see  Maltz  and 
Meyer,  2001;  Scallen  and  Hancock,  2001;  Schobel,  2009;  Schobel  and  Manzey,  2011;  Seppelt 
and  Lee,  2007;  Yamada  and  Kuchar,  2006).  Appropriate  trust  calibration  is  critical  for 
appropriate  reliance  on  automation  for  all  types  of  automated  aids.  According  to  Lee  and  Moray 
(1994),  trust  calibration  pertains  to  the  trust  level  of  the  human  user  and  its  correspondence  to  the 
capabilities  of  the  automated  aid.  As  such,  too  much  trust  may  lead  to  overreliance, 
complacency,  and  mistrust,  while  too  little  trust  can  lead  to  disuse.  Properly  calibrated  trust 
creates  a  situation  where  the  human  user  is  engaged  with  the  automation  and  participates  at  the 
appropriate  level  in  the  process  (Muir,  1994).  Calibration  of  trust  is  also  an  important  issue  for 
computer  users  in  the  e-commerce  domain.  Bonsall  and  Joint  (1991)  suggested  that  users  have 
difficulties  placing  the  “right  level”  of  trust  in  computer  advice  systems.  Computer  users  also 
have  difficulties  heeding  advice  from  a  computer  aid  during  the  decision-making  process 
(Hoffman  et  ah,  1999;  Ratnasingham,  1998). 

1,2  Definitions 

For  the  purpose  of  this  work,  it  is  important  to  understand  the  relationship  between  the  terms 
autonomy,  automation,  and  robot.  Literature  and  human  perception  have  often  blurred  the 
definitional  lines  between  these  terms.  For  example,  automation  and  autonomy,  as  well  as 
automation  and  robot,  are  often  used  interchangeably.  Here,  autonomy  is  defined  as  “a  capability 
(or  set  of  capabilities)  that  enables  a  particular  action  of  a  system  to  be  automatic  or,  within 
programmed  boundaries,  ‘self-governing’”  (DSB,  2012).  Automation,  in  the  broadest  sense  of 
the  word,  is  a  “machine  execution  of  functions”  (Parasuraman  et  ah,  2000,  p.  286).  With  such  a 
broad  definition  of  automation,  it  is  no  wonder  there  is  confusion  or  overlap  among  the 
definitions  of  autonomy,  automation,  and  robot.  In  addition,  a  large  portion  of  automation-based 
definitions  and  theoretical  contributions  build  on  the  work  by  Victor  Riley  (1989)  that  discussed 
automation  states  in  terms  of  combinations  of  values  along  the  dimensions  of  machine 
intelligence  (i.e.,  degree  of  data  processing)  and  autonomy  (i.e.,  degree  of  authority  to  act  on  the 
“World”).  For  example,  Parasuraman  and  Riley  (1997)  incorporated  the  human-machine 
comparison  in  which  automation  was  defined  as  “the  execution  by  a  machine  agent  (usually  a 
computer)  of  a  function  that  was  previously  carried  out  by  a  human”  (p.  231).  In  a  more  detailed 
view  of  the  functions  of  the  technology,  Lee  and  See  (2004)  define  automation  as  the 
“technology  that  actively  selects  data,  transforms  information,  makes  decisions,  or  controls 
processes”  (p.  50). 
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A  robot  is  often  defined  and  discussed  in  terms  of  its  functional  and  autonomous  capabilities. 
This  is  illustrated  by  Mahoney’s  (1997)  definition  of  a  robot  as  “programmable  automation  to 
augment  human  manipulation”  (p.  3).  More  recently,  Yagoda  (2011)  defined  a  robot  as  the 
interaction  of  intelligence  and  autonomy,  thus  building  on  Riley’s  (1989)  automation  research 
and  further  linking  the  two  definitions.  However,  additional  work  has  also  shown  the  importance 
of  physical  form  or  embodiment  to  the  classification  of  a  machine  as  a  robot.  For  example, 
Kurfess  (2005)  suggested  that  a  robot  is  the  manifestation  of  human  characteristics,  such  as 
complex  actions  and  anthropomorphic  features.  Schaefer  (2013)  also  provided  a  more  complete 
review  of  the  variations  between  robot  definitions  as  well  as  additional  experimentation  on  a 
human’s  perception  of  the  classification  of  a  machine  as  a  robot. 

In  addition,  Scholtz  (2003)  has  suggested  that  the  type  of  interaction,  the  number  of  systems  a 
user  may  interact  with,  the  environment,  and  the  physical  nature  of  the  robot  (e.g.,  complex, 
dynamic  control  systems,  exhibition  of  autonomy  and  cognition)  help  differentiate  HRI  from 
more  general  human-computer  or  human-machine  interaction.  Steinfield  et  al.  (2006)  suggested 
that  HRI  can  and  should  be  discussed  in  terms  of  five  types  of  tasks:  navigation,  perception  (i.e., 
perceive  and  understand  the  remote  environment  for  application),  management  (i.e.,  coordination 
of  the  actions  of  humans  and  robots),  manipulation  (i.e.,  interaction  with  the  environment),  and 
social  tasks  (i.e.,  perform  tasks  that  require  “social  interaction”).  Therefore,  by  taking  into 
account  the  task-based  nature  of  HRI,  it  may  be  more  accurate  to  classify  a  robot  as  having 
varying  or  multiple  autonomous  capabilities  within  a  single  embodiment  or  system. 

1,3  Report  Organization 

Section  2  identifies  three  types  of  task-specific  automation  (i.e.,  cognitive,  control,  and 
perceptual  aids)  and  the  importance  to  trust  development  within  each  type  of  automation  aid. 
Section  3  outlines  a  descriptive  model  of  human-automation  trust  and  provides  literary  support 
for  the  model  structure  in  terms  of  the  human-related,  automation-related,  and  environmental- 
related  trust  factors.  Section  4  outlines  the  analytical  methodology  for  the  meta-analytic  analysis 
provided  in  section  5.  Section  6  reconnects  the  human-automation  trust  results  with  the  potential 
links  within  HRI.  Training  suggestions,  as  well  as  design  recommendations,  are  provided.  This 
report  is  summated  with  a  conclusion  (section  7)  addressing  the  importance  of  this  work  to  the 
military  domain. 
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2.  Task-Specific  Trust  in  Automation 


Automation  literature  provides  a  number  of  insights  into  types  of  funetions  that  may  be 
automated  across  a  variety  of  tasks.  More  importantly,  task-  and  function-specific  aspects  of 
automation  play  an  important  role  in  the  development  of  trust  in  these  systems.  Task-related 
automation  is  hereby  broken  down  into  three  major  areas  of  automation  aids:  cognitive,  control, 
and  perceptual  aids.  This  accords  with  the  framework  discussed  in  Parasuraman  et  al.  (2000)  that 
identified  types  of  automation  as  acquisition,  analysis,  decision,  and  action  aids  while  taking  into 
account  that  the  task-related  nature  of  automation  elements  is  related  to  HRI. 

2.1  Trust  in  Cognitive  Aids 

Automated  cognitive  aids  provide  recommendations  to  users  about  the  current  and  potential 
future  states  of  systems.  These  automated  decision  aids  or  expert  systems  can  make  accurate 
identifications  or  detections  during  a  screening  or  inspection,  monitoring,  or  route  planning 
tasks.  This  type  of  aid  has  been  used  in  many  areas,  including  e-commerce,  medication 
management  domains,  and  many  others  (table  1). 

In  addition  to  these  factors,  Corritore  et  al.  (2003)  suggest  that  computer  interface  design  criteria 
(e.g.,  ease  of  use,  good  visual  design  elements,  information  content,  and  reputation)  can  provide 
cues  to  the  degree  to  which  an  aid  will  be  perceived  as  trustworthy.  One  concern  about  cognitive 
aids  is  that  operators  may  trust  and  rely  excessively  on  the  automated  aid.  This  can  lead  to 
reduced  monitoring  and,  in  turn,  increase  the  number  of  errors  (e.g.,  decision-making  task  for 
pilots,  Sarter  and  Schroeder,  2001).  Similar  findings  were  also  shown  by  Bagheri  and  Jamieson 
(2004)  during  a  monitoring  task:  increased  trust  in  the  automation  led  to  lower  operator  error 
detection. 
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Table  1.  Task-dependent  antecedents  of  trust  in  cognitive  automation. 


Task 

Antecedent 
of  Trnst 

Finding 

Target  Detection 

(Chen  and  Terrence,  2009; 
Wang  et  al.,  2009) 

Reliability  of 
automation 

A  more  reliable  aid  led  to  more  trust;  however,  this  effect  can  be 
moderated  by  individual  differences  factors,  such  as  operator 
attentional  control. 

Inspection  Task 

(Madhavan  and 

Wiegmann,  2007) 

Expertise  of 
Automation 

Users  trust  expert  automation  more  than  novice  automation. 

Human  vs. 
automation 

Users  have  more  trust  in  a  novice  automated  aid  than  a  novice 
human,  but  more  trust  in  an  expert  human  than  an  expert 
automated  aid. 

Task 

difficulty 

Users  trust  an  automated  aid  more  when  it  is  difficult  to  identify 
errors  in  the  task. 

Ronte  Planning 

(Chen  and  Barnes,  2012; 
Cummings  et  al.,  2010; 

De  Vries  and  Midden, 

2008) 

Error  rate  of 
automation 

When  automation  error  rate  is  high,  operators  trust  themselves 
more  than  the  automation;  conversely,  when  error  rate  is  low,  they 
trust  automation  more.  However,  these  effects  can  be  moderated 
by  individual  differences  factors,  such  as  operator  attentional 
control. 

Risk 

Participants  who  chose  less  risky  paths  trusted  the  automation 
more. 

Reputation 

Operators  had  slightly  higher  trust  when  the  automation  had  a 
positive  reputation. 

e-commerce 

(Aberg  and  Shahmehri, 
2000) 

Human  vs. 
automation 

Users  preferred  human  assistance  to  automation. 

Other  Decision  Aids 

(Ezer  et  al.,  2008; 

Ho  et  al.,  2005) 

Reliability  of 
automation 

A  more  reliable  aid  led  to  more  trust  in  the  aid. 

Age  of  user 

Older  adults  trusted  automation  more  than  younger  adults; 
however,  younger  adults  are  more  flexible  than  older  adults  in 
changing  their  reliance  behavior  based  on  the  cost  of  verification 
(of  the  automation). 
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2,2  Trust  in  Control  Aids 


Automated  control  aids  (e.g.,  autopilots,  adaptive  cruise  control,  and  navigation  aids)  replace 
varying  levels  of  human  (operator  or  user)  action.  Operators  tend  to  prefer  automated  control 
aids  to  traditional  systems  or  procedures  (Ardern-Jones  et  ah,  2009).  Trust  is  generally  high  for 
control  aids.  Initial  trust  tends  to  be  higher  for  the  automated  aid  (e.g.,  function  allocation  tasks, 
Ma  and  Kaber,  2007)  and  builds  over  time  (Rajaonah,  2006).  For  a  summary  of  factors 
impacting  trust  development  of  functional  allocation  tasks  and  vehicle  control,  see  table  2. 

Table  2.  Task-dependent  antecedents  of  trust  in  control  automation. 


Task 

Antecedent  of  Trust 

Finding 

Reliability 

Higher  reliability  led  to  higher  trust. 

Function  Allocation 

(Moray  et  at,  2000;  Stedmon 
et  al.,  2007) 

Communication 

mode 

Operators  had  higher  trust  for  automation  that  used 
human  speech  rather  than  synthetic  speech. 

Communication 

accuracy 

Operators  had  higher  trust  for  accurate  information 
than  inaccurate  information. 

Age  of  user 

Older  drivers  trusted  adaptive  cruise  control  (ACC) 
more  than  middle  age  drivers. 

Automation  accuracy 

Drivers  trusted  more  accurate  systems. 

Vehicle  Control:  ACC,  UAV 
(Donmez  et  at,  2006;  Hughes 
et  al.,  2009;  Spain  and  Bliss, 
2008) 

Alarms 

Drivers  trusted  visual  alarms  more  than  auditory 
alarms  for  vehicle  control. 

Perceived  usefulness 
of  automation 

As  the  level  of  usefulness  of  automation  increased,  so 
did  trust. 

Human  vs. 
automation 

Users  trusted  pilot  more  than  autopilot. 

Communication 

Higher  reliability  of  information  led  to  higher  trust. 

Workload 

Trust  was  higher  when  workload  was  low. 

Trust  also  has  a  direct  impact  on  outcome  measures.  For  example,  Manzey  and  his  colleagues 
(2009)  found  that  most  surgeons  trust  in  the  proper  functioning  of  the  navigational  control  tasks 
in  a  surgery  system  and  would  choose  to  be  operated  on  by  the  system.  More  operator  trust 
during  function  allocation  tasks  leads  to  more  use,  better  performance,  and  less  operator  time 
manually  adjusting  the  automation  (Muir  and  Moray,  1996).  In  addition,  higher  reliability  of 
adaptive  cruise  control  (ACC)  systems  leads  to  more  driver  trust  and  increased  compliance  with 
the  automation  (Spain  and  Bliss,  2008). 
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2,3  Trust  in  Perceptual  Aids 


Perceptual  aids  are  used  to  assist  the  operator  or  user  by  providing  warnings  (e.g.,  fault  detection, 
automobile  warning  systems,  and  function  allocation  for  signaling  or  communication)  or  to  assist 
with  pattern  recognition  (e.g.,  speech  or  target  detection).  For  example,  automobile  warning 
systems  are  used  for  distraction  mitigation,  collision  avoidance,  and  providing  alarms  (e.g., 
speed  alerts).  Gupta  and  colleagues  (2002)  assessed  the  effects  of  alarm  type  and  alarm 
sensitivity  (i.e.,  number  of  alarms)  on  drivers’  trust  of  perception  aids.  They  found  no  significant 
difference  in  type  of  alarm;  however,  drivers  did  show  a  higher  trust  for  alarms  with  a  low 
sensitivity  rate.  Similarly  to  the  other  two  types  of  automation  aids,  users  or  operators  of 
perception  aids  have  more  trust  in  people  than  in  the  automation.  For  example,  Bisantz  and 
Seong  (2001)  found  that  drivers  using  automobile  warning  systems  had  higher  negative  feelings 
of  trust  toward  the  automation  when  they  were  told  that  the  failures  were  attributed  to  the 
hardware  or  software  instead  of  people. 

Trust  in  the  warnings  or  alarms  also  has  a  direct  effect  on  the  outcome  variables.  One  potential 
area  for  concern  is  the  operator’s  overreliance  on  and  over  compliance  with  a  warning  system. 
Bitan  and  Meyer  (2007)  found  that  the  impacts  of  the  sensitivity  rate  and  the  false  alarm  rate  on 
trust  also  have  a  direct  impact  on  the  user’s  reliance  and  compliance  (respectively)  on  the 
warning  system  to  the  degree  in  which  the  operator  will  only  perform  an  action  when  an  alarm  is 
present  and  fail  to  respond  when  the  alarm  is  not  present. 


3.  Development  of  a  Trust-in-Automation  Model 


While  trust  is  important  to  specific  automated  tasks  and  outcomes  (especially  use  and 
performance),  the  factors  that  make  up  these  relationships  are  of  critical  importance  to  the 
development  or  degradation  of  trust  in  automation.  However,  despite  the  number  of  trust 
definitions  available,  there  is  a  lack  of  consensus  among  experts.  Billings  and  colleagues  (2012) 
reviewed  282  definitions  of  trust  (i.e.,  200  interpersonal  trust  definitions,  32  human-robot  trust 
definitions,  and  50  trust-in-automation  definitions).  These  automation  trust  definitions  highlight 
a  number  of  antecedents  that  are  important  to  trust  development  (see  figure  2)  and  support  the 
need  for  a  meta-analytic  approach  to  understanding  trust  in  automation. 
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Possible  antecendents  of  trust  found  in  definitions 

Figure  2.  Antecedents  of  trust  from  definitions  of  trust  in  automation. 


Trust  definitions  within  the  automation  domain  range  from  a  broad  definition,  “a  human’s 
willingness  to  accept  direction  from  an  automated  system”  (Fan  et  ah,  1998)  to  the  specific,  “the 
extent  to  which  a  user  is  confident  in,  and  willing  to  act  on  the  basis  of  recommendations, 
actions,  and  decisions  of  an  artificially  intelligent  agent”  (Madsen  and  Gregor,  2000,  p.  1).  Refer 
to  appendix  A  for  a  list  of  definitions  associated  with  human-automation  interaction,  and 
appendix  B  for  a  complete  list  of  trust-in- automation  definitions.  This  diversity  within 
definitions  is  seen  in  the  other  domains  of  trust  as  well,  supporting  the  idea  that  there  is  no 
universal  definition  of  trust.  Flowever,  one  of  the  most  referenced  definitions  of  trust  in 
automation  is  “the  attitude  that  an  agent  will  help  achieve  an  individual’s  goals  in  a  situation 
characterized  by  uncertainty  and  vulnerability”  (Lee  and  See,  2004,  p.  51). 

3,1  Theoretical  Model  of  Human-Automation  Trust 

To  identify  potential  antecedents  of  trust  development  in  human-automation  interaction,  existing 
literature  on  trust  in  automation  (both  theoretical  and  empirical)  was  reviewed.  The  literature 
suggests  that  several  factors  play  important  roles  in  trust  development.  Some  factors  involve  the 
human  directly,  while  others  focus  on  aspects  of  the  automation.  Other  factors  deal  with  various 
aspects  of  the  operational  environment.  Therefore,  the  antecedents  of  trust  are  hereby  divided  by 
three  main  factors:  the  human,  the  automation,  and  the  environment  (see  figure  3).  The  following 
section  will  analyze  the  empirical  evidence  of  human-automation  trust  through  meta-analytic 
procedures.  The  findings  from  this  analysis  will  then  be  used  to  update  the  model  of  human- 
robot  trust. 
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Human  Operator 

Automation 

Environment 

Human  Traits 

Features 

Task-Related 

-f*  Age 

*  Degrees  of  automation 

-(-*  Proximity 

+Gender 

-(-*  Appearance 

-FRisk 

-(-Ethnicity 

-(-Personality 

*  Mode  of  communication 

Human  States 

Capability 

Fatigue 

-(-*  Errors  in  automation 

Stress 

-(-*  Automation  behavior(s) 

Attentional  control 

Quality/ Accuracy 
*  Cueing/Feedback/Alarms 

Cognitive  Factors 

-(-*  Understanding  the  automation 

-(-*  Ability  to  use  automation 
-(-*  Expectancy  of  automation 

Emotive  Factors 

-(-Attitudes  toward  automation 

Confidence  in  automation 
-(-Satisfaction  with  automation 
-(-*  Comfort  with  automation 

Note:  *  represents  experimental  and  +  represents  correlation  findings. 

Figure  3.  Theoretical  model  of  human-automation  trust. 

3.2  Factors  Associated  With  the  Human 

As  seen  within  human-teehnology  interaetion,  the  human  eomponent  is  often  suggested  to  be  of 
utmost  importance;  however,  the  factors  associated  with  the  human  are  often  overlooked  in 
empirical  research  (as  seen  by  the  limited  number  of  cited  work)  or  become  a  secondary  focus 
behind  the  functional  capabilities  of  the  system.  Here  we  discuss  trust  as  a  human-centric 
construct  in  that  the  human  is  the  one  trusting  the  technology.  Therefore,  the  human  user  or 
operator  must  be  considered  when  examining  trust.  Within  the  area  of  trust  in  automation,  the 
factors  associated  with  the  human  can  be  addressed  in  terms  of  human  traits  and  states  as  well  as 
the  cognitive  and  emotive  factors. 

3,2,1  Operator  Traits  and  States 

There  is  evidence  that  stable  human  traits  may  be  related  to  how  an  individual  responds  to 
automation,  but  it  is  likely  that  their  relationships  vary  as  a  function  of  the  characteristics  of  the 
automated  task  (Szalma  and  Taylor,  2011).  For  example,  many  demographic  characteristics  of 
the  user  or  operator  have  been  suggested  to  influence  the  human-automation  trust  relationship, 
including  age,  gender,  ethnicity,  and  personality.  For  example,  age  can  impact  the  types  of 
systems  that  will  be  trusted  (e.g.,  driver  warning  systems,  Kircher  and  Thorslund,  2009)  and 
influence  the  likelihood  that  the  automation  will  be  used  (e.g.,  decision  aids  for  medication 
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management,  Ho  et  al.,  2005).  Ho  and  eolleagues  also  found  that  older  adults  were  more  likely  to 
trust  the  automation  than  younger  adults.  However,  upon  further  exploration,  gender  differenees 
do  not  seem  to  influenee  trust  development  (Merritt  and  Ilgen,  2008;  Stedmon  et  ah,  2007). 

Some  work  suggests  that  ethnieity  and  personality  traits  are  important  to  trust  development 
(Merritt  and  Ilgen,  2008).  However,  they  have  not  been  thoroughly  explored  in  the  automation 
trust  literature.  These  anteeedents  of  trust  are  more  eommonly  referenced  in  human-interpersonal 
trust  literature  and  are  therefore  included  in  figure  3. 

Dynamic  human  states,  such  as  stress  and  fatigue,  during  human-automation  interaction  are  well- 
researched  areas  (see  Finomore  et  ah,  2009;  Neubauer  et  ah,  2012;  Reichenbach  et  ah,  2011; 
Warm  et  ah,  2008).  For  the  purpose  of  this  work,  stress  is  the  “force  that  degrades  performance 
capability”  (Hancock  and  Warm,  1989).  Fatigue  is  a  “reduction  in  the  capacity  and  desire  to 
react .  .  .  characterized  by  tiredness  and  an  aversion  to  the  continuation  of  goal-directed  work” 
(Hancock  et  ah,  2012).  In  addition,  several  studies  also  showed  that  operators  with  lower 
attentional  control  interacted  differently  with  automated  systems  than  those  with  higher 
attentional  control  (Chen  and  Terrence,  2009;  Chen  and  Barnes,  2012;  Thropp,  2006). 

Attentional  control  is  defined  as  “one’s  ability  to  focus  and  shift  attention  in  a  flexible  manner” 
(Chen  and  Barnes,  2012;  Derryberry  and  Reed,  2002).  Participants  with  lower  attentional  control 
tended  to  rely  more  heavily  on  automation,  even  when  the  reliability  of  those  systems  was  low. 
This  research  indirectly  links  human  states  to  trust  development;  however,  additional  empirical 
study  is  needed  in  these  areas. 

3.2,2  Cognitive  Factors 

The  operator’s  cognitive  factors  are  hereby  divided  into  three  areas  of  research:  (1) 
understanding  how  the  automation  works,  (2)  ability  to  use  the  automation,  and  (3)  expectancy 
of  the  automation.  While  these  are  most  often  researched  in  the  interpersonal  trust  domain, 
Ashleigh  and  Stanton  (2001)  suggest  that  these  areas  of  research  are  applicable  to  the  technology 
domain  as  well.  In  addition,  these  divisions  are  supported  by  previous  work  in  the  field  of  trust  in 
automation  (see  Lee  and  See,  2004;  Madhavan  and  Wiegmann,  2007). 

3.2.2. 1  Understanding  How  the  Automation  Works 

Understanding  how  to  operate  the  automation  is  based  in  part  on  the  ease  of  learning  the  system 
and  an  individual’s  previous  experiences.  Investigations  into  ease  of  learning  have  yielded  mixed 
results.  While  one  study  (Wang  et  ah,  2011)  indicates  that  trust  is  negatively  correlated  with  ease 
of  learning,  others  have  shown  that  a  novice  operator’s  trust  grows  along  a  typical  learning  curve 
with  training  (Muir  and  Moray  1996),  and  it  is  critical  to  learning  (Swadesh  and  Kasonf,  1995). 
Past  experiences  with  technology,  specifically  automated  systems,  have  a  direct  impact  on  trust 
development  as  well  (Merritt  and  Ilgin,  2008).  For  example,  Rajaonah  (2006)  suggests  that  trust 
is  something  that  builds  over  time  and  use  with  a  system.  Work  by  Sullivan  and  colleagues 
(2008)  supports  the  idea  that  trust  builds  over  time  and  suggests  that  prior  experiences  directly 
influence  how  an  operator  will  engage  the  automation.  In  the  computer  domain,  research  has 
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shown  that  with  sustained  exposure  to  and  use  of  information  teehnologies,  eomputer  users 
develop  trust  in  the  aeeuraey  of  the  data  stored  within  a  eomputer  system  despite  the  faet  that  there 
is  upwards  of  10%  inaeouraeies  due  to  error  (Klein  et  ah,  1997;  Parasuraman,  1987;  Wiener  1985). 

3. 2. 2.2  Ability  to  Use  the  Automation 

Another  area  of  the  operator’s  eognitive  faetors  that  affeets  the  human- automation  trust 
relationship  is  the  operator’s  pereeived  ability  to  use  automation.  This  ineludes  self-effioaey, 
workload,  and  expertise  as  anteeedents  of  trust.  Self-effioaey  is  often  defined  as  an  individual’s 
pereeived  abilities  to  suooessfully  oarry  out  an  identifiable  task  (Bandura,  1982;  1997).  In  the 
field  of  automation,  self-effioaey  affeets  not  only  whether  or  not  a  human  will  use  automation, 
but  also  how  the  human  will  peroeive  ease  of  use  and  usefulness  (Igbaria  and  livari  1995). 

One  purpose  of  automation  is  to  reduoe  workload,  and  this  has  been  shown  to  be  suooessful  in 
both  mental  workload  (Wiegmann  et  ah,  2010)  and  in  the  implementation  of  notions  (Yang  et  ah, 
2009).  Inoreases  in  mental  workload  assooiated  with  the  automated  systems  have  been  shown  to 
lead  to  degradations  of  trust  in  automation  (e.g.,  oombat  identifioation  tasks,  Wang  et  ah,  2011). 
In  their  work  on  adaptive  aiding  of  human-robot  teaming,  de  Visser  and  Parasuraman  (2011) 
found  that  human-robot  teams  benefited  from  imperfeot  statio  automation  in  high-task-load 
eonditions,  and  adaptive  automation  helped  alleviate  the  workload  of  the  operator. 

Workload  has  also  been  direetly  tied  to  expertise  and  experienee  with  automation.  For  example, 
in  a  surgieal  task  using  image-guided  navigation,  inexperieneed  users  reported  inereased  time 
pressure  and  mental  demands  when  using  the  systems  (Manzey  et  ah,  2009).  Expertise  may  also 
be  inhibitory,  espeeially  in  highly  reliable  systems  where  a  higher  experienee  level  may  impair 
an  operator’s  ability  to  monitor  for  unantieipated  system  states  (Bailey  et  ah,  2006).  As  sueh, 
inereased  expertise  with  a  system  ean  enhanee  trust  development  (Rajaonah,  2006). 

3. 2. 2. 3  Expeetaney  of  the  Automation 

Prior  to  interaeting  with  automation,  human  operators  develop  mental  models  or  expeetations  of 
how  the  automation  should  behave.  Therefore,  this  subeomponent  of  eognitive  faetors  ineludes 
the  following  anteeedents  of  trust:  pereeived  eommitment,  usefulness,  expeetations,  and  benefits 
of  automation.  Keil  et  al.  (1995)  found  that  the  pereeived  usefulness  (e.g.,  number  of  antieipated 
errors)  of  an  automated  system  is  often  dietated  by  a  funetion  of  the  task  or  the  fit  of  tool.  The 
pereeived  benefit  of  the  automation  ean  infiuenee  whether  or  not  the  automation  will  be  trusted 
(similar  to  software  systems,  Eee  et  ah,  2007).  If  those  expeetations  are  high,  the  human  operator 
may  beeome  overreliant  on  the  automation  (Mayer  et  ah,  2006),  whieh  ean  lead  to  mistrust  and 
misuse  of  the  system.  The  system’s  reputation  may  impaet  operator  expeetations.  Eor  example, 
in  an  experiment  involving  a  planning  aid,  De  Vries  and  Midden  (2008)  found  trust  to  be  slightly 
higher  when  the  aid  was  known  to  have  a  positive  reputation,  bindings  also  suggest  that  the 
number  of  people  (i.e.,  majority  or  minority)  supporting  the  reputation  did  not  appear  to 
infiuenee  trust  development. 
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3.2,3  Emotive  Factors 

Emotive  factors  can  be  divided  into  four  areas  of  research:  attitudes  toward  automation, 
confidence  in  automation,  satisfaction,  and  comfort  with  automation.  Similar  to  the  previously 
listed  cognitive  factors,  emotive  factors  are  most  often  researched  in  the  interpersonal  trust 
domain. 

3.2.3. 1  Attitudes  Toward  Automation 

Attitudes  toward  the  automation  can  impact  overall  system  awareness  states.  Bailey  and 
colleagues  (2006)  found  that  operators  with  positive  attitudes  toward  automation  may  have  too 
much  trust  in  the  system  and  focus  less  on  the  overall  system  state,  which  can  lead  to  degraded 
performance.  Ardern-Jones  and  colleagues  (2009)  found  that  errors  in  the  system  or  difficulty 
accessing  needed  information  within  the  system  can  lead  to  negative  attitudes  toward 
automation.  Gao  et  al.  (2006)  found  that  operators  with  negative  attitudes  toward  the  automation 
(such  as  after  a  failure)  were  more  likely  to  disuse  the  automation.  Additionally,  moods  and 
emotions  impact  trust  and  liking  of  a  system  (e.g.,  happiness,  Merritt  et  ah,  2012)  as  well  as 
reliance  on  automated  systems. 

3. 2. 3. 2  Confidence  in  Automation 

Confidence  is  often  used  as  a  synonym  for  trust.  For  example,  in  their  evaluation  of  a  navigation 
system  (NaviBase)  for  Ears,  Nose,  and  Throat  surgery,  Strauss  and  colleagues  (2006)  evaluated 
trust  in  the  system  by  measuring  user  confidence.  Other  researchers  suggest  that  confidence  is 
one  of  the  antecedents  leading  to  trust  development.  Operators  need  to  be  confident  in  the 
functionality  of  the  automation  to  successfully  interact  and  use  said  automation.  Gao  et  al.  (2006) 
found  that  an  operator  will  switch  to  manual  control  when  confidence-based  trust  in  automation 
decreases.  Operators  are  also  more  likely  to  remain  in  manual  control  in  the  future,  even  when  it 
is  clear  that  the  automation  is  more  efficient  than  human  performance.  Therefore,  we  are 
including  confidence  as  an  antecedent  of  trust. 

There  is  also  debate  regarding  whether  self-confidence  should  be  included  in  a  human- 
automation  trust  model.  Many  researchers  suggest  that  there  is  an  interaction  between  trust  and 
self-confidence  as  it  pertains  to  automation  use  (see  Chen  and  Terrence,  2009;  De  Vries  et  ah, 
2003;  Fee  and  Moray,  1994;  Neyedli  et  ah,  2009;  Wang  et  ah,  2011).  However,  authors  disagree 
on  the  interpretation  of  this  interaction.  Fee  and  Moray  (1994)  suggest  that  it  is  the  relationship 
between  trust  and  self-confidence  that  leads  to  automation  use  (i.e.,  trust  exceeds  self- 
confidence)  or  manual  override  control  (i.e.,  self-confidence  overrides  trust).  Rani  et  al.  (2000) 
suggest  that  operators  who  have  both  high  trust  and  high  self-confidence  tend  to  prefer  a  higher 
level  of  automation.  Since  there  is  a  lack  of  agreement  regarding  the  relationship  between  trust 
and  self-confidence,  self-confidence  will  only  be  discussed  here  and  will  not  be  included  in  the 
meta-analysis  results  that  follow. 


13 


3. 2. 3. 3  Satisfaction  With  Automation 


As  is  the  case  in  many  other  arenas,  satisfaction  influences  human  perception  of  automation. 

This  satisfaction  is  fostered  by  the  quality  of  information  and  service  that  the  automation 
provides  (Lee  et  ah,  2007).  Generally,  as  satisfaction  increases,  so  does  trust  (Domnez  et  ah, 
2006).  In  one  example,  Wang  and  colleagues  (2011)  found  trust  to  be  positively  correlated  with 
satisfaction  in  a  combat  identification  task. 

3. 2. 3. 4  Comfort  With  Automation 

Whether  or  not  users  are  comfortable  with  an  automated  aid  may  influence  their  perception  of 
the  aid  and,  ultimately,  whether  or  not  they  trust  the  aid.  The  level  of  comfort  an  individual  has 
with  automation  may  be  dependent  on  familiarity  and  proximity  (Nakada,  1997;  van  den  Broek 
and  Westerink,  2009),  similarity  of  intent  (Verberne  et  ah,  2012),  or  even  the  degree  of  control 
the  automated  system  has  over  levels  of  control,  tactics,  or  strategy  of  the  task  (Ward,  2000). 

The  majority  of  automated  systems  function  within  close  proximity  to  their  operators.  Interaction 
typically  occurs  through  a  computer-based  interface.  To  improve  vehicle  user  interface  design 
for  optimum  safety,  Takayama  and  Nass  (2008)  examined  the  role  of  computer  proximity  (in-car 
versus  wireless)  on  driving  behavior.  They  suggest  that  in-car  automated  systems  may  be  more 
effective  for  local  information  (e.g.,  climate,  car  maintenance),  while  distant  computers 
connected  by  a  wireless  connection  may  be  better  for  global  information  (e.g.,  navigational 
instructions).  Additionally,  systems  showed  that  drivers  felt  more  engaged  with  the  in-car 
system,  potentially  leading  to  safer  driving  behavior. 

3,3  Factors  Associated  With  the  Automation 

The  primary  trust  development  factors  associated  with  the  automation  itself  include  features  and 
capability  of  the  automation.  Here  we  discuss  the  features  of  the  automation  in  terms  of  the 
degrees  of  automation  (e.g.,  level  of  automation),  appearance,  and  mode  of  communication.  The 
capability  of  the  automation  is  discussed  in  terms  of  errors,  behavior,  quality/accuracy,  and 
communication  through  cueing  and  feedback. 

3,3,1  Feature-Based  Characteristics 

Operators  tend  to  have  a  fundamental  bias  toward  trust  in  one’s  own  abilities  or  other  human’s 
abilities  rather  than  technology  or  automated  systems  (see  Dzindolet  et  ah,  2003;  Gao  et  ah, 

2006;  Hughes  et  ah,  2009).  However,  research  by  Ma  and  Kaber  (2007)  suggest  that  initial  trust 
can  be  higher  for  the  automated  control  aid  than  a  human  performing  the  same  function.  In 
addition,  Madhavan  and  Wiegmann  (2007)  found  that  trust  is  higher  with  novice  automation 
(e.g.,  provides  general  assistance)  than  a  novice  human,  while  trust  is  higher  with  an  expert 
human  than  expert  automation  (e.g.,  provides  specific  or  detailed  assistance).  The  following 
features-based  antecedents  of  trust  can  assist  in  understanding  trust  development  in  automation. 
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3. 3. 1.1  Degrees  of  Autonomy 

This  subeomponent  of  automation  features  ineludes  levels  of  automation  (LOA)  and  mode  of 
automation  (e.g.,  fixed,  adjustable,  and  adaptive  automation)  as  antecedents  of  trust.  The  LOA 
can  range  from  high  to  low,  with  a  high  score  indicative  of  an  automated  system  where  the 
computer  or  system  is  totally  autonomous  and  the  human  has  no  control,  and  low  score  for  a 
system  where  the  human  is  totally  in  control  and  the  computer  offers  no  assistance  (for  a 
thorough  review  of  the  levels  of  automation,  see  Parasuraman  et  ah,  2000).  While  Rani  et  al. 
(2000)  found  no  interaction  between  trust  and  the  preferred  LOA,  more  recent  work  suggests  that 
LOA  has  a  direct  impact  on  trust  development.  For  example,  the  combination  of  multiple  types 
of  automation  to  provide  adaptive  assistance  is  more  trusted  than  individual  lower  LOAs  (Cai 
and  Lin,  2012;  Cai  et  ah,  2012).  In  addition,  Merritt  and  colleagues  (2012)  suggest  that  when  an 
individual  implicitly  perceives  automation  as  “good,”  the  automated  system  is  more  trusted  than 
if  it  is  perceived  as  having  “poor”  or  “ambiguous”  automation. 

LOA  also  ties  directly  into  research  on  the  mode  of  automation  (e.g.,  fixed,  adjustable,  and 
adaptive)  through  the  degree  to  which  the  human  interacts  with  the  automation.  Adjustable 
automation  is  a  direct  interaction  with  automation  in  which  the  user  or  operator  has  more  direct 
control  over  the  automation.  Adaptive  automation  is  an  indirect  interaction  with  the  automation 
that  takes  into  account  the  operator  (e.g.,  emotional  state,  performance,  and  workload)  and  adapts 
the  type  and  amount  of  automation  accordingly  (e.g.,  emotion-aware  consumer  products,  van  den 
Brock  and  Westerink,  2009).  Human  operators  typically  trust  manual  adjustable  automation 
more  than  adaptive  automation  (e.g.,  function  allocation  tasks,  Moray  et  ah,  2000)  because  of  the 
direct  human  interaction.  However,  Looije  et  al.  (2010)  suggested  that  when  interacting  with 
adaptive  automation,  humans  prefer  adaptive  automation  that  can  learn,  recognize,  and  respond 
to  personality  differences  in  humans  as  well  as  exhibiting  specific  personality  traits  itself. 

3. 3. 1.2  Appearance 

If  an  automated  aid  or  machine  is  perceived  to  be  aesthetically  compatible  with  its  function,  it  is 
more  likely  to  be  used  (Goetz  et  al.,  2003).  More  specifically,  Goetz  and  colleagues  suggest  that 
when  the  appearance  of  an  automated  aid  matches  the  user’s  expectations,  the  automated  aid  is 
perceived  more  positively,  and  the  user  is  more  likely  to  appraise  it  positively.  Further,  general 
likeability  has  been  shown  to  be  influenced  by  appearance  (Li  and  Yeh,  2010).  Nass  and  Moon 
(2000)  also  suggest  the  importance  of  anthropomorphic  appearance  on  automated  systems.  Pak 
and  colleagues  (2012)  found  that  increasing  the  physical  anthropomorphic  appearance  of  an 
automated  aid  significantly  impacted  trust  development  through  subjective  assessment  as  well  as 
obj  ective/behavioral  assessment. 

3. 3. 1.3  Mode  of  Communication 

Mode  of  communication  refers  to  the  human  sensory  system  that  the  automation  cues  (e.g., 
visual,  auditory,  and  tactile).  Sarter  (2006)  reviewed  guidelines  focusing  on  multimodal 
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information  presentation  and  the  researeh  needed  in  the  area,  emphasizing  the  need  to  design  the 
various  parameters  earefully  to  ensure  their  eonsisteney  with  the  users’  personality  to  maximize 
trust.  For  example,  specific  experimentation  on  auditory  communication  demonstrated  that 
operators  tend  to  trust  systems  with  human  speech  more  than  synthetic  speech  and  showed 
enhanced  performance  into  the  human  speech  condition  (Stedmon  et  ah,  2007). 

3.3.2  Automation  Capability 

The  capability  of  automation  is  one  of  the  most  researched  areas  of  automation  trust  literature. 
While  capabilities  are  typically  task  specific,  three  major  areas  that  can  be  discussed  include: 
errors,  behavior,  and  cueing  or  feedback. 

3.3.2. 1  Errors 

Errors  are  unavoidable  in  human-automation  interaction;  however,  the  goal  of  design  is  to  reduce 
the  number  of  errors.  Errors  are  traditionally  discussed  in  terms  of  both  misses  and  false  alarms 
(see  Dixon  and  Wickens,  2006;  Dixon  et  ah,  2007);  however,  trust  literature  suggests  that  the 
reduction  of  all  types  of  errors  increases  the  reliability  of  the  system  and  positively  impacts  trust 
development  (De  Vries  et  ah,  2003).  As  such,  the  level  of  trust  is  mainly  based  on  the  user’s 
perception  of  the  competence  of  the  automated  aid  (Muir  and  Moray,  1996).  Higher  reliability 
has  also  been  shown  to  lead  to  higher  operator  response  frequency  (i.e.,  an  indirect  measure  of 
trust,  Spain  and  Bliss,  2008)  and  higher  reliance  on  the  automation. 

In  one  study  with  conflicting  findings,  Uggirala  and  colleagues  (2004)  found  that  the  number  of 
errors  presented  by  a  system  had  no  effect  on  trust,  but  the  authors  suggested  that  this  was 
probably  because  the  number  of  correct  and  incorrect  responses  was  held  constant  for  each 
treatment.  Therefore,  we  typically  perceive  errors  as  negative  influences  on  trust  development  in 
that  when  automation  has  too  high  of  an  error  rate,  operators  will  not  trust  the  automation  and 
disuse  it.  However,  some  errors  may  assist  in  trust  calibration  to  reduce  overreliance  and  over¬ 
trust  in  the  system,  which  leads  to  reduced  monitoring  of  the  automation  and  negative 
performance. 

The  rate  of  errors  is  also  important  in  the  human-automation  trust  relationship.  Yamada  and 
Kuchar  (2006)  found  that  in  drivers,  miss  rate  was  related  to  the  degradation  of  trust  in  a  system, 
which  led  to  slower  response  times.  Conversely,  it  is  the  degree  of  difficulty  of  the  task  and  not 
the  type  of  error  that  appears  to  influence  trust  level.  In  a  target  identification  task,  Madhavan 
and  Wiegmann  (2007)  found  no  significant  difference  between  an  easy  miss  group  and  an  easy 
false  alarm  group;  however,  the  difficult  errors  group  trusted  the  automation  more  than  the  easy 
errors  group.  Similar  findings  by  Master  and  colleagues  (2005)  showed  significant  change  in 
trust  between  conservative  and  risky  systems  but  no  difference  in  trust  based  on  error  rates.  This 
suggests  that  individuals  are  more  willing  to  trust  automation  despite  errors  when  the  task  is 
difficult. 
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In  general,  operators  and  users  of  automated  systems  have  enhaneed  trust  in  the  automation 
when  information  is  aeeurate  and  reliable  (Spain  and  Bliss,  2008;  Stedmon  et  ah,  2007). 
Degraded  performanee  has  also  been  shown  when  inaeeurate  information  was  presented  (Sarter 
and  Sehroeder,  2001).  When  multiple  automation  aids  are  used  simultaneously,  the  potential  for 
eonfheting  information  to  be  transmitted  by  different  automated  alerting  systems  ean  inerease.  In 
addition.  Song  and  Kuehar  (2003)  found  that  this  eonflieting  information  ean  lead  to  distrust  of 
one  or  all  of  the  automated  systems.  Therefore,  the  aeeuraey  of  the  eommunieation  beeomes 
inereasingly  important  to  trust-related  responses. 

3. 3. 2.2  Behavior 

Similar  to  human- interpersonal  trust,  the  behavior  of  automation  influenees  a  user’s  pereeived 
trust  in  the  automated  aid.  Importantly,  trust  in  automation  is  established  by  eonsistent  and 
reliable  behavior  on  the  part  of  the  automation.  Over  time,  this  trust  reduees  the  effort  expended 
by  the  human  user,  beeause  as  trust  is  gained,  the  foeus  shifts  away  from  observing  speeifie 
behaviors  toward  assessing  the  global  disposition  of  the  system  (MeLeod  et  ah,  2005). 

Therefore,  this  subeomponent  of  automation  eapability  ineludes  the  following  anteeedents  of 
trust:  predietability,  dependability,  and  speeifie  types  of  behaviors. 

The  quality  of  the  automation  ineludes  automation  expertise,  etiquette,  and  transpareney  of 
eommunieation  and  behavior.  Researeh  shows  that  users  trust  speeialized  (expert)  automation 
more  than  they  do  general-purpose  automated  deviees  or  automation  they  pereeive  to  be  noviee 
(Koh  and  Sundar,  2010;  Madhavan  and  Wiegmann,  2007).  Etiquette  of  automation  refers  to  a 
eolleetion  of  behaviors  that  indieate  that  the  system  ean  be  deemed  trustworthy  or  displays 
behaviors  that  are  satisfying  in  aeeordanee  to  the  eustoms  of  a  eulture  (Parasuraman  and  Miller, 
2004).  Current  literature  suggests  that  etiquette  is  related  to  trust  and  ean  lead  to  a  higher 
probability  of  intent  errors  (Dzindolet  et  ah,  2006;  Lee  and  Moray,  1992;  Lee  and  See,  2004; 
Lees  and  Lee,  2007;  Merritt  and  Ilgen,  2008;  Parasuraman  and  Miller,  2004).  This  researeh 
suggests  that  trust-related  etiquette  ean  inerease  the  probability  that  an  operator  will  disregard 
what  they  know  about  the  utility  of  the  automation  when  deeiding  to  use  or  not  use  the 
automation.  Simply  put,  enhaneing  automation  with  etiquette  features  inereases  trust  in 
automation,  despite  previous  knowledge  or  reputation. 

Predietability  and  dependability  of  the  automation  are  some  of  the  most  well-researehed  areas  of 
trust  in  automation  work.  Predietability  refers  to  how  well  the  behaviors  of  the  automation  mateh 
the  operator’s  expeetation  of  that  behavior.  Dependability  refers  to  the  eonsisteney  and 
effeetiveness  of  a  behavior.  Systems  with  high  reliability  engender  the  trust  of  their  operator 
aeross  eontrol,  eognitive,  and  pereeptual  task  types  (Bagheri  and  Jamieson,  2004;  Bailey  et  ah, 
2006;  Bliss  and  Aeton,  2003;  Cahour  and  Lorzy,  2009;  Cummings  et  ah,  2010;  Donmez  et  ah, 
2006;  Ho  et  ah,  2005;  Kazi  et  ah,  2007;  Moray  et  ah,  2000).  Similarly,  predietability  influenees 
user  trust.  If  experienee  with  a  maehine  provides  predietable  outeomes,  then  operators  may  start 
to  trust  the  system  (Cahour  and  Lorzy,  2009;  Muir,  1994).  However,  when  operators  experienee 
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unanticipated  reactions  from  the  automation  (e.g.,  unpredictable  outcomes),  there  is  a  rapid  drop 
in  trust  that  often  leads  to  disuse  or  a  disregard  for  future  information  provided  by  the  aid 
(Wiegmann  et  ah,  2010).  Dependable  systems,  speeifieally  the  effectiveness  of  a  warning 
system,  are  strongly  related  to  the  operator’s  trust  (Yamada  and  Kuchar,  2006).  When  examining 
the  relationship  of  pilots  and  their  automated  aids,  Tenney  et  al.  (1998)  found  that  pilots  wanted 
the  automation  to  be  dependable  and  provide  information  that  was  simple  and  informative.  These 
qualities  were  ranked  more  important  than  aecountability  (able  to  explain  its  actions), 
subordination  (easily  eountermanded),  flexibility  (range  of  modes  and  levels),  adaptability 
(reprogrammable),  error  resistance  (refuse  erroneous  inputs),  or  error  tolerance  (compensate  for 
erroneous  inputs)  of  automation  behaviors. 

3. 3. 2. 3  Cueing  and  Feedbaek 

Many  automated  systems  provide  some  level  of  cueing  and/or  feedbaek  to  the  operator.  The 
appropriateness  of  ones  and  feedbaek  has  been  shown  to  have  a  direct  connection  to  trust 
development.  In  a  study  examining  the  implications  of  installing  ACC  systems  in  vehicles,  it  was 
found  that  the  development  of  the  driver’s  trust  in  the  automated  system  may  depend  on  the 
system’s  ability  to  effeetively  communicate  the  status  of  the  ACC  to  drivers  to  determine  when 
intervention  is  necessary  (Stanton  et  ah,  2007).  Bitan  and  Meyer  (2007)  suggest  that  the 
availability  of  information  is  important  to  trust  development.  For  example,  it  was  suggested  that 
providing  drivers  with  continuous  information  about  the  state  of  the  automation  may  provide  a 
necessary  alternative  to  providing  imminent  erash  warnings  when  it  fails  (Seppelt  and  Lee, 

2007).  Further,  the  amount  of  feedback  sought  from  an  automated  system  by  the  operators  is 
directly  related  to  the  degree  of  trust  they  have  that  it  will  perform  without  failure  (Muir  and 
Moray,  1996). 

Different  types  of  cueing  may  be  used  to  provide  the  user  with  feedback.  An  experiment  that 
investigated  the  impact  of  automation  on  young  and  older  adults’  abilities  to  detect  threat  objeets 
found  that  spatial  cueing  (e.g.,  highlighting  a  speeific  target)  improved  performanee  for  both  age 
groups;  conversely,  text  cueing  helped  young  adults  but  had  no  impact  on  older  adults’ 
performance  (Wiegmann  et  ah,  2006).  Sharpies  and  colleagues  (2007)  identified  the  impact  of 
different  types  of  eommunication  (i.e.,  text,  radio,  faee-to-face)  on  trust  development,  in  whieh 
true  commands  are  trusted  more  than  false  eommands.  Alarms  ean  be  designed  to  provide  very 
specific  types  of  feedback  to  the  operator.  It  is  important  that  the  user  understand  the  necessity 
and  benefit  of  the  alarm.  The  effect  of  the  type  of  alarm  on  trust  development  appears  to  be  task 
dependent.  For  example,  Gupta  and  colleagues  (2002)  found  no  signifieant  difference  in  trust  by 
type  of  alarm  in  a  collision  avoidance  system,  yet  Donmez  and  eolleagues  (2006)  found  that 
drivers  trust  visual  more  than  auditory  alarms  for  distraction  mitigation.  The  sensitivity  of 
alarms — specifically,  the  frequency  and  duration — also  affects  trust  development  and  use.  Gupta 
and  colleagues  (2002)  found  that  a  reduced  number  of  alarms  led  to  more  trust  in  a  collision 
avoidance  system.  Wickens  and  colleagues  (2009)  support  this  finding  by  suggesting  that 
operators  perceive  exeessive  alarms  to  eontain  false  alerts  and  are  thereby  unneeessary.  This 
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leads  to  distrust  in  the  automation  and  an  operator  disregard  for  true  alerts.  However,  Lees  and 
Lee  (2007)  found  an  interesting  distinetion  between  false  alarms  and  unneeessary  alarms.  They 
found  that  while  false  alarms  deereased  trust  and  complianee  levels,  unnecessary  alarms 
(instances  where  the  automated  device  indicates  a  hazardous  situations,  but  the  human  may  not 
view  them  as  hazardous)  actually  increase  trust  and  compliance  levels. 

3,4  Factors  Associated  With  the  Environment 

While  factors  specific  to  the  human  and  the  automation  are  important  in  trust  development,  there 
are  also  factors  specific  to  the  environment.  These  environment-based  factors  often  require  an 
interaction  between  the  human  and  automation  within  a  specific  context.  Trust  is  often  a  task- 
dependent  construct  (refer  to  section  2).  The  task  domain  and  the  environment  that  it  is 
completed  in  can  impact  trust  development.  Risk  and  uncertainty  are  of  essential  importance  in 
trust  development.  Here  we  discuss  risk  in  terms  of  both  the  risk  of  the  automation  system  as 
well  as  the  risk  from  the  environment.  For  example.  Heckles  et  al.  (2005)  looked  at  the  internal 
risk  of  data  corruption  from  the  automation  of  a  computerized  credential  management  system. 
They  suggest  that  the  system  will  not  be  trusted  if  the  information  is  either  compromised  or 
revoked.  Lyons  et  al.  (2011)  used  factor  analysis  to  examine  trust  in  the  context  of  IT  suspicion 
and  determined  trust  and  distrust  to  be  orthogonal  constructs,  and  levels  of  trust  are  dependent 
upon  cues  in  the  environment.  Horst  et  al.  (2010)  investigated  automation-based  pilot  alerting 
systems  for  the  external  risk  of  the  physical  terrain.  They  suggest  that  trust  is  an  important 
element  in  these  types  of  automated  systems  because  of  the  safety  risk  associated  with  this  type 
of  human-automation  interaction. 


4.  Analytical  Method 


4.1  Sample  of  Studies 

A  review  of  empirical  and  nonempirical  articles  dealing  specifically  with  human-automation 
trust  was  conducted  using  Web  of  Science  and  EHSCOhost  databases  with  automation  and  trust 
as  the  primary  search  terms.  We  also  used  Google  and  its  derivative  Google  Scholar  to  perform 
searches  for  the  primary  search  terms.  After  an  initial  listing  of  articles  was  obtained,  the 
references  for  these  papers  were  reviewed  to  determine  whether  any  other  related  studies  could 
be  identified.  This  process  resulted  in  343  articles  and  conference  proceedings  published 
between  1960  and  2012.  Hoth  empirical  studies  and  nonempirical  reports  were  collected  initially. 

4.2  Criteria  for  Study  Inclusion 

All  343  identified  studies  were  inspected  to  ensure  that  they  fulfilled  the  following  four  criteria 
for  inclusion  in  the  meta- analysis: 
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1 .  Each  study  had  to  report  an  empirical  examination  of  trust  in  which  trust  was  a  directly 
measured  outcome  of  an  experimental  manipulation.  Studies  in  which  trust  served  as  the 
experimental  manipulation  were  excluded. 

2.  The  empirical  examination  of  trust  was  directed  toward  automation.  Thus,  studies 
investigating  interpersonal  trust  and  human-robot  trust  were  excluded.  Empirical  studies 
that  focused  on  trust  in  automated  computer-based  systems  were  included  in  this  analysis. 

3.  Each  included  study  was  required  to  incorporate  human  participants  who  either  viewed  or 
participated  directly  in  interactions  with  the  automation  through  physical,  virtual,  or 
augmented  means. 

4.  Each  study  must  include  sufficient  information  to  determine  effect  size  estimates.  Authors 
of  potentially  relevant  statistics  were  contacted  for  additional  statistical  data  needed  to 
calculate  effect  size. 

Of  the  original  343  articles  on  human-automation  trust  that  were  collected,  287  were 
immediately  omitted  because  of  failure  to  meet  the  criteria  for  inclusion  in  the  meta- analysis. 
Eirst,  12  articles  were  written  in  another  language  other  than  English  or  could  not  be  located. 
Second,  57  articles  investigated  trust  in  something  other  than  automation  (e.g.,  financial  trusts, 
interpersonal  trust,  robotics).  Third,  218  articles  did  not  directly  measure  trust  and  were  excluded 
from  the  analysis.  Of  these  discarded  articles,  126  discussed  theoretical  underpinnings  for  trust 
and  potential  antecedents  of  trust  in  automation,  48  introduced  factors  involved  in  human- 
automation  interaction,  and  44  discussed  programming,  automation  design,  and/or  development. 

A  total  of  56  empirical  articles  were  identified;  however,  27  of  these  articles  had  insufficient 
information  to  calculate  effect  size.  Authors  were  contacted  via  email  and  were  given  5  weeks  to 
respond  with  additional  information.  Of  these  studies,  14  were  not  included  in  the  final  analysis. 
It  is  important  to  note  that  rejecting  primary  studies  in  a  meta- analysis  is  a  common  occurrence 
and  is  necessary  to  ensure  meaningful  results  when  combining  effect  sizes  across  studies. 

The  42  papers  meeting  the  inclusion  criteria  are  identified  in  appendix  C  of  this  report  by  either 
an  asterisk  (*)  or  a  cross  (+)  appearing  in  front  of  the  first  author’s  name.  The  asterisk  (*) 
represents  those  studies  included  in  the  experimental  analyses.  The  cross  (+)  represents  those 
studies  included  in  the  correlational  analyses  (see  also  appendix  D). 

4,3  Coding  of  Studies 

Available  study  characteristics  from  each  experiment  were  coded,  including,  but  not  limited,  to 
the  following;  dependent  variable(s),  independent  variable(s),  statistical  analysis  used,  test 
values,  degrees  of  freedom,  means,  standard  deviations,  alpha  level,  and  statistical  significance. 
If  more  than  one  variable  was  manipulated  in  a  study,  each  independent  variable  was  coded  and 
analyzed  separately.  Several  additional  variables  were  also  coded,  including  the  following:  (1) 
type  of  task,  (2)  participant  population,  (3)  type  of  trust  (e.g.,  trustworthiness,  general  trust),  and 
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(4)  main  moderator  (human,  automation,  or  environment),  submoderator  (refer  to  figure  4),  and 
anteeedent  of  trust  (deseribed  in  seetion  3).  Partieipant  population  and  type  of  trust  were  not 
ineluded  for  analysis  beeause  of  the  laek  of  variation  between  studies. 

4.4  Effect  Size 

The  studies  ineluded  in  effeet  size  ealeulation  eontained  both  eorrelational  and  group  design 
data,  therefore  the  use  of  multiple  meta-analytie  methods  (Pearson’s  r  and  Hedges’  g)  was 
neeessary.  The  eorrelational  effeets  represent  an  assoeiation  between  trust  and  the  given  faetor. 
Hedges’s  g  indieates  the  standard  differenee  between  two  means  in  standard  deviation  units. 
From  these  we  ean  gather  eorrelational  and  eausal  inferenees  between  trust  and  any  given  faetor. 
Through  both  types  of  meta-analytie  effeets,  the  more  positive  the  effeet  represents  a  higher  level 
of  trust.  Findings  were  interpreted  using  Hedges’  g  and  Pearson’s  r  established  ranges  for  small 
(g  <  .20,  r  <  .10),  medium  (g  =  .50,  r  =.25),  and  large  (g  >  .80,  r  >  .40)  effeet  sizes. 

4.5  Variance  Estimates 

Several  varianee  estimates  were  ealeulated.  First,  variability  of  the  effeet  sizes  themselves  (s  g) 
and  variability  due  to  sampling  error  (s  e)  were  estimated.  Next,  these  two  values  were  used  to 
eompute  the  residual  varianee  (s  s).  A  large  (s  s)  is  an  indieation  that  the  effeet  sizes  may  be 
heterogeneous,  and  therefore  one  or  more  variables  are  likely  to  be  moderating  the  magnitude  of 
that  partieular  effeet.  A  final  eheek  for  homogeneity  of  varianee  (^^52)  was  ealeulated  (proportion 

of  total  varianee  aeeounted  for  by  sampling  error).  Hunter  and  Sehmidt  (2004)  suggest  that  an 
outeome  here  of  0.75  or  greater  implies  that  the  remaining  varianee  is  due  to  statistieal  artifaets, 
and  sueh  oases  are  therefore  interpreted  as  indioating  homogeneity  of  varianee  among  the  effeet 
sizes.  However,  large  residual  varianee  and  small  homogeneity  of  varianee  may  also  be  due  to  a 
small  number  of  studies,  as  is  evident  in  some  of  the  following  results. 


5.  Results 


The  42  papers  meeting  the  inolusion  oriteria  yielded  a  total  of  166  experimental  effeet  sizes  and 
63  eorrelational  effeet  sizes.  Following  organization  of  the  studies,  three  main  types  of  analyses 
were  eondueted:  trusts  preferenee  for  human  or  automation  assistanee  (seetion  5.1),  trust  in  type 
of  task  by  eognitive,  eontrol,  and  pereeptual  aids  (seetion  5.2),  and  moderator  analyses  (seetion 
5.3  and  5.4).  Moderator  analyses  ean  be  direetly  linked  to  figure  3  diseussing  the  human-, 
automation-,  and  environmental-related  faetors  of  trust. 

5,1  Preference  for  Human  or  Automation  Assistance 

There  is  debate  in  the  literature  as  to  how  humans  trust  teehnology.  While  some  literature  has 
suggested  that  human  users  of  teehnology  tend  to  trust  themselves  or  other  humans  more  than 
teehnology,  others  suggest  the  exaet  opposite  (see  Dzindolet  et  ah,  2001;  Lyons  and  Stokes, 
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2012).  Three  empirieal  studies  on  how  humans  trust  in  automation,  yielding  nine  statisties,  were 
obtained  (see  De  Vries  et  ah,  2003;  Ma  and  Kaber,  2007;  Madhavan  and  Wiegmann,  2007).  The 
Lyons  and  Stokes  (2012)  study  was  not  retained,  as  it  measured  relianee,  not  speeifically  trust. 
Similarly,  the  study  by  Dzindolet  and  her  colleagues  (2001)  was  not  retained,  as  it  focused  on 
disuse  and  misuse  of  automation,  and  not  directly  trust  development. 

Results  (g  =  -1.30)  support  the  finding  that  operators  of  automation  tend  to  trust  themselves  or 
other  humans  more  than  the  technology  (table  3).  Although  the  results  suggest  large  variability  in 
the  effect  sizes  themselves  and  sampling  error,  the  proportion  of  total  variance  accounted  for  by 
sampling  error  (+.75)  may  be  indicative  of  the  small  number  of  studies.  There  were  no 
correlation-based  analyses  conducted  for  this  factor.  This  finding  is  important  to  keep  in  mind 
when  designing  automated  systems. 

Table  3.  Meta-analytic  results  for  perception  of  trust  in  a  human  or  automation  for  a  task. 


Category  k 

9 

si  si  isl 

95%  Cl 

Lower  Upper 

N 

Human  vs.  Automation  3 

-1.30 

.81  .61  .21  .75 

-2.18  -.41 

143 

k  =  number  of  studies. 

N  =  sample  size. 

s^g  estimates  the  variability  of  the  effect  sizes  themselves, 
s^e  estimates  the  variability  due  to  sampling  error. 
s^5  is  an  estimate  of  the  residual  variance. 

(s^e !  s^g)  is  a  calculation  of  homogeneity  of  variance. 


5,2  Task-related  Automation 

While  30  experimental  studies  yielding  164  experimental  effect  sizes  met  the  original  selection 
criteria,  this  analysis  only  included  studies  that  were  specifically  categorized  by  the  task-related 
type  of  automation  (i.e.,  cognitive,  perceptual,  or  control  aids).  Therefore,  27  experimental 
studies,  which  yielded  155  experimental  effect  sizes,  were  included  for  analysis.  These  results, 
shown  in  table  4,  indicated  there  was  a  moderate  global  effect  concerning  trust  and  automation 
{g  =  +0.48).  As  the  confidence  interval  here  excluded  zero,  we  can  assume  this  is  a  substantive 
and  consistent  effect.  The  subdivision  of  this  global  effect  into  the  three  types  of  task-related 
automation  indicated  that  the  cognitive  (13  studies,  72  statistics),  perceptual  (6  studies,  14 
statistics),  and  control  (8  studies,  68  statistics)  aids  all  had  a  moderate  effect  on  trust 
development.  All  findings  appear  to  be  consistent;  however,  the  check  for  the  homogeneity  of 
variance  for  the  cognitive  aids  suggests  that  the  remaining  variance  may  be  due  to  statistical 
artifacts.  This  finding  allowed  us  to  complete  all  additional  analysis  as  a  complete  sample 
without  the  need  to  consider  task  type  as  a  moderating  variable. 
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Table  4.  Meta-analytic  results  for  trust  in  task-based  automation. 


Category 

k 

9 

^9 

4 

Se/ 

si 

95%  Cl 

N 

Lower 

Upper 

Type  of  Automation 

27 

-H.48 

.38 

.26 

.12 

.69 

.28 

.68 

999 

Cognitive  Aids 

13 

+41 

.20 

.20 

.00 

1.00 

.14 

.67 

533 

Perceptual  Aids 

6 

+62 

.63 

.30 

.33 

.48 

.18 

1.06 

134 

Control  Aids 

8 

+51 

.54 

.36 

.18 

.67 

.09 

.93 

333 

k  =  number  of  studies. 

N  =  sample  size. 

s^g  estimates  the  variability  of  the  effect  sizes  themselves, 
s^e  estimates  the  variability  due  to  sampling  error. 
s^5  is  an  estimate  of  the  residual  variance. 

(s^e !  is  a  calculation  of  homogeneity  of  variance. 


Correlation  analyses,  ineluding  16  studies  and  63  effeet  sizes,  were  also  eondueted.  Ten  studies 
(43  effect  sizes)  were  included  in  cognitive  aids,  yielding  a  moderate  to  large  effect  (f  =  +0.39) 
on  trust  development.  Perceptual  aids  yielded  a  moderate  effect  (f  =  +0.37)  on  trust 
development;  however,  this  result  was  limited  to  one  study.  Five  studies  (19  effect  sizes)  were 
included  in  control  aids,  yielding  a  small  effect  on  trust  development  (f  =  +0.12). 

5,3  Human-Related  Factors 

Theoretical  work  across  technological  domains  suggests  the  importance  of  the  human  element  in 
trust  development.  Nine  studies  (54  statistics)  were  included  for  analysis,  with  three  being 
specific  to  age,  four  specific  to  cognitive  factors,  and  three  specific  to  emotive  factors.  Overall, 
the  human  element  has  a  moderate  effect  {g  =  +0.49)  on  trust  (table  5).  When  looking 
specifically  at  the  submoderating  effects  of  age,  we  found  a  moderate  effect  {g  =  +0.44)  on  trust 
development.  The  results  of  these  three  studies  suggest  that  automation  is  trusted  more  by  older 
adults  than  by  younger  adults.  Only  one  study  discussed  the  correlation  between  age  and  trust,  so 
no  additional  analyses  were  conducted.  The  submoderating  effect  of  cognitive  factors  suggests  a 
small  to  medium  effect  {g  =  +0.39).  Additional  research  should  be  conducted  in  this  area  to 
solidify  the  effects  of  cognitive  traits  on  trust  development.  Three  experimental  studies  on 
emotive  factors  of  trust  development  yielded  a  moderate  to  large  effect  {g  =  +0.72)  on  trust 
development.  Six  studies,  yielding  1 1  effect  sizes,  were  included  for  correlation-based  analysis 
of  emotive  factors.  This  yielded  a  small  to  medium  effect  (f  =  +0.16),  suggesting  that  trust  and 
emotive  factors  are  indeed  related,  albeit  small. 
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Table  5.  Meta-analytic  results  for  human-related  factors  on  trust  in  automation. 


Category 

k 

9 

^9 

4 

4! 

95%  Cl 

Lower  Upper 

N 

Human 

9 

+A9 

.08 

.13 

-.05 

1.70 

.25 

.72 

395 

Traits  (Age) 

3 

+.44 

.15 

.09 

.06 

.63 

.09 

.79 

143 

Cognitive  Factors 

4 

+39 

.01 

.15 

-.14 

15.39 

.01 

.77 

162 

Emotive  Factors 

3 

+.72 

.01 

.18 

-.17 

21.18 

.24 

1.20 

104 

k  =  number  of  studies. 

N  =  sample  size. 

s^g  estimates  the  variability  of  the  effect  sizes  themselves, 
s^e  estimates  the  variability  due  to  sampling  error. 
s^5  is  an  estimate  of  the  residual  variance. 

(s^e !  s^b)  is  a  calculation  of  homogeneity  of  variance. 


5.4  Automation-Related  Factors 

Automation-related  factors  were  the  most  researched  area  of  trust  in  automation  with  22  studies 
(100  statistics).  Overall  analysis  of  the  automation-related  factor  suggests  a  moderate  effect  {g  = 
+0.53)  on  trust  development  (table  6). 


Table  6.  Meta-analytic  results  for  automation-related  factors  on  trust  in  automation. 


Category 

k 

9 

^9 

4/ 

Ai 

95%  Cl 

Lower  Upper 

N 

Automation 

22 

+.53 

.52 

.31 

.21 

.59 

.29 

.78 

766 

Features 

8 

+.35 

.17 

.36 

-.19 

2.12 

-.10 

.79 

270 

Capability 

16 

+.70 

.87 

.30 

.57 

.35 

.41 

.99 

517 

Behavior 

10 

+.79 

.50 

.29 

.22 

.57 

.44 

1.15 

437 

Error 

3 

+.44 

.36 

.09 

.28 

.24 

.11 

.77 

140 

Feedback 

6 

+.45 

1.93 

.82 

1.11 

.43 

-.28 

1.18 

140 

k  =  number  of  studies. 

N  =  sample  size. 

s^g  estimates  the  variability  of  the  effect  sizes  themselves, 
s^e  estimates  the  variability  due  to  sampling  error. 
s^5  is  an  estimate  of  the  residual  variance. 

(s^e !  s^g)  is  a  calculation  of  homogeneity  of  variance. 

A  more  detailed  look  at  the  submoderating  factors  of  features  and  capabilities  provides  a  more 
detailed  understanding  of  the  findings.  Literature  suggests  a  theoretical  importance  of  automation 
features  on  trust  development.  However,  the  effects  of  correlation  analysis  based  on  only  two 
studies  (f  =  +0.16)  and  experimental  analysis  {g  =  +0.35)  include  zero  in  the  confidence 
interval,  suggesting  an  unstable  effect.  Analyses  of  automation  capabilities  showed  a  moderate 
correlational  effect  on  trust  (f  =  +0.39)  and  moderate  to  large  experimental  effect  {g  =  +0.70)  on 
trust  development.  The  antecedents  of  the  automation  capabilities,  behavior  {g  =  +.79)  and  error 
{g  =  +0.44),  were  also  analyzed.  These  results  suggest  that  the  capability  of  the  automation  is 
important  to  trust  development  in  that  the  behaviors  should  be  reliable  and  that  lower  error  rate 
leads  to  more  trust  in  the  system. 
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6.  Discussion 


Roboticists  are  more  recently  attempting  to  extend  the  classification  of  a  robot  beyond  that  of 
traditional  automation  to  include  the  importance  of  cognition  and  intelligence  (see  Fong  et  ah, 
2001;  Yagoda,  2011),  as  well  as  the  environment  (see  Groom,  2008;  Lin  et  ah,  2008).  The  types 
of  interaetion,  the  number  of  systems  a  user  may  interact  with,  the  environment,  and  the  physieal 
nature  of  the  robots  (e.g.,  complex,  dynamie  control  systems,  exhibition  of  autonomy  and 
eognition)  further  differentiate  HRI  from  human-computer  or  human-machine  interaetion 
(Scholtz,  2003).  However,  the  area  of  automation  and  trust  in  automation  overlaps  with  human- 
robot  trust  to  some  degree  and  can  help  us  better  understand  human-robot  trust  and  extend  the 
human  robot  trust  model  first  discussed  in  Hancock  et  al.  (2011,  see  also  figure  1).  Figure  4 
represents  the  updated  descriptive  model  of  human-robot  trust  that  takes  into  account  the 
findings  from  this  human-automation  trust  meta-analysis. 


Human 

Robot 

Environment 

Demographic  Traits 

Features 

Task-Related 

Age 

Levels  of  Automation 

Ri  sk/U  nc  ertainty 

Gender 

Appearance/ Anthropomorphism 

Physical  Environment 

Ethnicity/Culture 

Type 

Task  Type/Multitasking 

Personality 

Robot  Personality 

States 

Communication  Mode 

Team  Collaboration 

Fatigue 

Capability 

In-group  Membership 

Stress 

Reliability/Errors 

Shared  Mental  Models 

Attentional  Control 

Behaviors(s) 

Cognitive  Factors 

Understanding 

Ability  to  use/ Attention 
Expectancy 

Emotive  Factors 

Attitudes 

Confidence 

Satisfaction 

Comfort 

Feedback/Cueing/Alarms 

Figure  4.  Revised  human-robot  descriptive  trust  model. 

A  number  of  changes  were  made  from  the  original  deseriptive  trust  model.  The  human-related 
factor  was  reorganized  from  ability-based  eharacteristics  and  human  characteristics  subheadings 
to  more  specifie  subheadings  (i.e.,  demographic  traits,  human  states,  eognitive  faetors,  and 
emotive  factors).  The  robot-related  factor  was  reorganized  from  performanee-based  and 
attribute-based  subheadings  to  features  and  capabilities.  Findings  from  this  work  provided 
additional  support  for  the  importanee  of  communication  on  trust  development.  Communication 
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was  also  moved  from  the  environmental-related  faetor  to  the  robot-related  factor  and  discussed 
in  terms  of  communication  mode  (robot  feature)  and  feedback  (robot  capability).  Overall 
findings  demonstrated  that  the  human  and  automation-related  factors  had  a  moderate  to  large 
effect  on  trust  development.  Findings  from  this  study  were  also  used  to  identify  potential  training 
and  design  benefits  specific  to  human-robot  interaction. 

6.1  Training  Suggestions 

1 .  Including  some  degree  of  error  or  faUure  during  training  may  be  important  in  trust 
calibration.  While  our  analysis  demonstrated  that  traditionally  a  low  error  rate  has  a 
moderate  effect  on  trust  development  {g  =  +0.44),  Bahner  and  colleagues  (2008)  suggested 
that  exposing  users  to  failures  during  training  can  be  used  to  reduce  complacency  and 
automation  bias.  An  over-trusting  operator  is  more  likely  to  become  complacent  and  follow 
the  suggestions  of  the  automation  without  cross-checking  the  validity  against  other 
available  and  accessible  information.  Automation  bias  is  the  operator’s  tendency  to  trust 
the  automation  even  when  contradictory  information  is  present  or  when  information 
provided  by  the  aid  is  known  to  be  incorrect. 

2.  Scenario  training  may  decrease  the  operators  ’  unwillingness  to  use  automation. 
Automation  has  been  shown  to  provide  benefits  to  tasks  that  were  traditionally 
accomplished  by  humans.  Our  meta- analytic  findings  suggest  that  despite  these  known 
benefits,  operators  or  users  may  tend  to  trust  other  humans  or  themselves  more  than 
automation  {g  =  -1.30).  Scenario  training  may  assist  users  or  operators  by  allowing  them  to 
compare  the  utilities  of  automated  and  nonautomated  control  (Beck  et  ah,  2007),  which  can 
reduce  these  effects. 

3.  Training  can  provide  additional  understanding  of  the  system.  An  increase  in  automation  is 
predicted  for  the  future,  especially  in  the  robotic  domain.  Sharpies  et  al.  (2007)  suggest  that 
novel  technologies  for  flight  deck  air  traffic  control  systems  have  great  potential  for 
increasing  safety  when  the  system  is  understood  by  the  team  interacting  with  it.  Therefore, 
operators  and  users  need  additional  training  with  capabilities  of  the  systems.  Our  analysis 
demonstrated  that  automation  capabilities  {g  =  +0.70),  behaviors  effect  {g  =  +.79),  and 
communication  feedback  {g  =  +.45)  between  the  operator  and  automation  had  a  large  effect 
on  trust  development. 

6.2  Design  Recommendations 

1 .  LOAs  should  be  reviewed  to  determine  specifications  of  the  automation  for  a  given  task. 
Often  within  human-automation  interaction,  the  LOA  dictates  the  type  of  communication 
that  occurs.  Communication  in  terms  of  accuracy,  feedback,  cueing,  mode,  and  access  to 
information  was  shown  to  be  essential  to  trust  development.  Our  findings  suggested  that 
the  features  of  the  automation  including  LOA,  mode  of  communication  (g  =  +.35),  as  well 
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as  the  capability  of  the  automation  in  terms  of  communication  feedback  (g  =  +.45),  were 
each  shown  to  have  a  small  to  moderate  effect  on  trust  development. 

The  most  referenced  material  on  levels  of  automation  is  the  Parasuraman  et  al.  (2000) 
article  on  the  10  levels  of  automation  (figure  5).  Robots  tend  to  incorporate  multiple  types 
of  automation  across  cognitive,  perceptual,  and  control  aids.  Therefore,  these  10  levels  may 
provide  too  much  detail  to  successfully  incorporate  and  test  in  the  design.  One  possible 
solution  is  a  reduction  in  levels  to  include  human-in-the-loop  (manual  or  frequent  human 
interaction),  human-on-the-loop  (human  monitors  and  has  ability  to  intervene),  and 
autonomous  (no  human  intervention). 


High 

10. 

The  automation  decides  everything,  acts  autonomously,  ignoring  the  human. 

9. 

The  automation  informs  the  human  only  if  it,  the  robot,  decides  to 

8. 

The  automation  informs  the  human  only  if  asked 

7. 

The  automation  executes  automatically,  then  necessarily  informs  the  human 

6. 

The  automation  allows  the  human  a  restricted  time  to  veto  before  automatic  execution 

5. 

The  automation  executes  that  suggestion  if  the  human  approves 

4. 

The  automation  suggests  one  alternative 

3. 

The  automation  narrows  the  selection  down  to  a  few  options 

2. 

The  automation  offers  a  complete  set  of  decision/action  alternatives 

Low 

1. 

The  automation  offers  no  assistance:  the  human  must  make  all  decisions  and  actions 

Figure  5.  Levels  of  automation  (Parasuraman  et  al.,  2000). 

2.  Control  displays  should  be  appropriately  designed  to  enhance  communication.  Many 
different  sensors  are  utilized  in  robot  design  and  use  multiple  modalities  (visual,  auditory, 
tactile)  to  communicate  with  human  operators  or  users.  Reising  and  Sanderson  (2002) 
suggested  that  control  displays  should  be  appropriately  instrumented  and  designed 
according  to  the  principles  of  ecological  interface  design,  otherwise  they  might  lead  to 
misinterpretation.  Our  analysis  suggested  that  there  is  a  large  effect  of  communication  on 
trust  development;  however,  more  research  needs  to  occur  to  determine  the  correct 
communication  modality  based  on  task  needs  and  physical  environment.  In  addition,  more 
research  needs  to  be  conducted  on  the  effects  of  the  type  of  aid  (cognitive,  perceptual, 
control)  on  human-automation  trust. 

3.  Design  can  have  a  direct  impact  on  human  safety.  Semmer  (2005)  suggested  that  trust  can 
impact  how  operators  interact  with  automation  and  that  designers  should  include  the  social 
and  political  aspects  of  design  as  well  as  the  individualistic  needs.  Theoretical  work  in 
automation  suggests  the  importance  of  risk  and  uncertainty  in  trust  development;  however, 
empirical  literature  is  lacking  in  this  domain.  The  relationship  between  robot  design  and 
safety  becomes  an  even  more  important  area  of  research  as  robotic  capabilities  continue  to 
advance  through  increased  interdependency  of  multiple  types  of  automation,  as  well  as  the 
societal  need  for  robots  to  interact  in  a  wide  variety  of  physical  environments. 
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7.  Conclusions 


Across  the  years  and  especially  in  the  more  reeent  deeades,  we  have  seen  the  general  functions 
of  the  military  beeome  ever  more  subject  to  technological  innovation.  This  includes  all  aspects  of 
the  military  mission,  but  it  is  most  especially  evident  in  combat  and  conflict  studies.  All  trends 
look  toward  a  “distancing”  from  immediate  interaction  in  which  the  agency  of  such  distancing  is 
technology.  Not  solely  a  mediating  influenee,  advaneed  technologies  promise  to  beeome  a 
determining  influence  in  whieh  all  decisions  we  advaneed  to  are  thus  promulgated  by 
“automation.”  That  those  systems  are  eurrently  simply  design  destinations  at  remote  points  in 
space  and  time  are  of  little  comfort  to  remaining  eombatants  and  to  pereeption  of  the  greater 
publie  in  general.  Meanwhile,  we  are  entering  into  a  phase  of  hybrid  eontrol  where  action  is 
effected  by  automation  but  mediated  and  monitored  by  remaining  humans-in-the-loop.  As  this 
latter  situation  is  an  interaetive  one,  we  have  to  look  to  examine  the  effectiveness  of  that 
interaction  and  the  factors  that  mediate  its  effect.  Here,  trust  is  a  vital  dimension.  A  system 
untrusted  is  a  system  unused  or,  worse,  is  a  system  misused.  Yet,  over-trust  brings  its  own 
inherent  problems.  While  our  overall  focus  is  on  the  Soldier-robot  relationship,  we  directly 
acknowledge  that  this  represents  a  subset  of  the  overall  Soldier-automation  relationship. 
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Attentional  Control:  One’s  ability  to  focus  and  shift  attention  in  a  flexible  manner  (Chen  and 
Barnes,  2012;  Derryberry  and  Reed,  2002). 

Automation:  (1)  Machine  execution  of  functions  (Parasuraman  et  ah,  2000);  (2)  “the  execution 
by  a  machine  agent  (usually  a  computer)  of  a  function  that  was  previously  carried  out  by  a 
human”  (Parasuraman  and  Riley,  1997,  p.  231);  (3)  “the  technology  that  actively  selects  data, 
transforms  information,  makes  decisions,  or  controls  processes”  (Lee  and  See,  2004,  p.  50). 

Cognitive  Aids:  Provide  recommendations  to  users  about  the  current  and  potential  future  states 
of  systems. 

Control  Aids:  Replace  varying  levels  of  human  (operator/user)  action. 

Perceptual  Aids:  Used  to  assist  the  operator  or  user  by  providing  warnings  or  to  assist  with 
pattern  recognition. 

Autonomy:  (1)  A  capability  (or  set  of  capabilities)  that  enables  a  particular  action  of  a  system  to 
be  automatic  or,  within  programmed  boundaries,  “self-governing”  (Defense  Science  Board, 
2012);  (2)  degree  of  authority  to  act  on  the  “World”  (Riley,  1989). 

Etiquette  (system):  A  collection  of  behaviors  that  indicate  that  the  system  can  be  deemed 
trustworthy,  or  displays  behaviors  that  are  satisfying  in  accordance  to  the  customs  of  a  culture 
(Parasuraman  and  Miller,  2004). 

Fatigue:  A  reduction  in  the  capacity  and  desire  to  react  (Hancock  et  ah,  2012). 

Robot:  (1)  “Programmable  automation  to  augment  human  manipulation”  (Mahoney,  1997,  p.  3); 
(2)  an  autonomous  system  that  acts  on  its  own  decisions  with  input  from  a  human  operator  but 
may  not  be  completely  controlled  by  a  human  (Feil-Seifer  and  Mataric,  2005);  (3)  the  interaction 
of  intelligence  and  autonomy  of  the  technology  (Yagoda,  2011);  (4)  a  complex,  dynamic  system 
that  shows  a  degree  of  autonomy  and  cognition  as  it  operates  in  a  real-world  environment  (Fong 
et  ah,  2001);  (5)  perceives  the  environment  through  sensor  inputs  that  may  or  may  not  require 
direct  human  response  and  has  the  ability  to  move  in  order  to  manipulate  an  uncertain  physical 
world  by  transmitting  information  at  humans  or  directly  acting  on  the  environment  (Groom, 
2008;  Lin  et  ah,  2008). 

Self-efficacy:  An  individuaFs  perceived  abilities  to  successfully  carry  out  an  identifiable  task 
(Bandura,  1982;  1997). 

Stress:  The  force  that  degrades  performance  capability  (Hancock  and  Warm,  1989). 
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Definition  Citation 

Definition 

Adams  and  Webb  (2003)*t* 

Trust  is  a  psychological  state  involving  positive  confident  expectations 
and  willingness  to  act  on  the  basis  of  these  expectations.  Issues  of  trust 
arise  in  contexts  that  involve  risk,  vulnerability,  uncertainty  and 
interdependence.  Trust  expectations  are  created  primarily  by  the 
interaction  of  the  perceived  qualities  of  the  trustee  and  contextual  factors 
in  play  when  trust  decisions  are  made  (p.  38). 

Barber,  B.  (1983) 

Barber  defines  trust  in  terms  of  a  taxonomy  of  three  specific  expectations 
that  can  be  extended  to  the  human/machine  relationship,  according  to  Muir 
(1994): 

•  our  general  expectation  of  the  persistence  of  the  natural  physical  order, 
the  natural  biological  order,  and  the  moral  social  order, 

•  our  specific  expectation  of  technically  competent  role  performance  from 
those 

involved  with  us  in  social  relationships  and  systems, 

•  our  specific  expectation  that  partners  in  an  interaction  will  carry  out  their 
fiduciary  obligations  and  responsibilities,  that  is,  their  duty  in  certain 
situations  to  place  others'  interests  before  their  own. 

Bentley  et  al.  (1995) 

Trust  and  familiarity  is  a  process  that  is  partially  a  function  of  technology 
(i.e.,  trust  and  familiarity  can  be  embedded  in  use)  (p.  151). 

Biros  et  al.  (2004) 

“System  automation  trust  is  defined  as  having  confidence  in  and  entrusting 
the  system  automation  to  do  the  appropriate  action”  (p.  177). 

In  this  model,  dependability  and  predictability  contribute  to  an  individual’s 
overall  trust  in  a  system’s  automation. 

Boon  and  Holmes  (1991) 

“A  state  involving  confident  predictions  about  another’s  motives  with 
respect  to  oneself  in  situations  entailing  risk.” 

Cohen  etal.  (1998) 

The  perceived  probability  of  the  system’s  reliability,  given  certain 
situations  (APT  model).  “The  problem  of  decision  aid  acceptance  is 
neither  under  trust  nor  over  trust,  but  inappropriate  trust:  a  failure  to 
understand  or  properly  evaluate  the  conditions  affecting  good  and  bad  aid 
performance”  (p.  1). 

Costa,  Roe,  and  Taillieu  (2001)<» 

“Trust  is  a  psychological  state  that  manifests  itself  in  the  behaviours 
towards  others,  is  based  on  the  expectations  made  upon  behaviours  of 
these  others,  and  on  the  perceived  motives  and  intentions  in  situations 
entailing  risk  for  the  relationship  with  those  others”  (p.  228). 

Curral  (1990)»> 

“Relying  on  someone  under  conditions  of  dependence  and  risk.” 

Curral  and  Inkpen  (2002)  ❖ 

Trust  is  the  decision  to  rely  on  a  partner  with  the  expectation  that  the 
partner  will  act  according  to  a  common  agreement. 

Deutsch  (1973)»> 

“The  confidence  that  one  will  find  what  is  desired  from  another,  rather 
than  what  is  feared”  (p.  148). 

Note:  References  marked  with  a  ❖  represent  definitions  that  are  originally  from  another  domain  of  trust  but  are  referenced  or 
applied  to  automation. 
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Definition  Citation 

Definition 

De  Vries  et  al.  (2003) 

“We  consider  trust  to  be  the  expectation  of  a  user  about  the  system,  that 
the  system  will  perform  a  certain  task  for  him  or  her,  while  the  outcome 
of  that  task  is  uncertain,  in  that  it  can  have  both  positive  and  negative 
consequences.  In  other  words,  an  element  of  risk  should  be  present” 

(p.  722). 

Fan  et  al.  (1998) 

A  human’s  willingness  to  accept  direction  from  an  automated  system. 

Jeannot  et  al.  (2003) 

“Trust  is  a  result  of  many  factors  such  as  reliability  of  the  system  and 
transparency  of  the  functions.  Neither  mistrust  nor  complacency  are 
desirable”  (p.  4). 

Lee  and  See  (2004) 

“Trust  can  be  defined  as  the  attitude  that  an  agent  will  help  achieve  an 
individual’s  goals  in  a  situation  characterized  by  uncertainty  and 
vulnerability”  (p.  51). 

Luhmann  (1979)»> 

Mechanism  to  reduce  feelings  of  uncertainty  via  a  continuous  feedback 
loop  the  person  monitors  the  behavior  of  the  trustee  to  determine  whether 
the  trust  vested  in  the  person  is  justified. 

Madhavan  and  Wiegmann  (2007) 

“Trust  refers  to  the  expectation  of,  or  confidence  in,  another  and  is  based 
on  the  probability  that  one  party  attaches  to  co-operative  or  favourable 
behaviour  by  other  parties  (Barber  1983,  Muir  1987,  Hwang  and 

Buergers  1997)”  (p.  280). 

Madsen  and  Gregor  (2000) 

Comprised  of  the  reliability  of  the  system,  perceived  technical 
competence  of  the  system,  perceived  understandability  of  the  system, 
faith  into  the  system  and  personal  attachment  to  the  system. 

Madsen  and  Gregor  (2000) 

Human-computer  trust  is  defined  in  this  study  to  be,  the  extent  to  which  a 
user  is  confident  in,  and  willing  to  act  on  the  basis  of  the 
recommendations,  actions,  and  decisions  of  an  artificially  intelligent 
agent. 

Master  et  al.  (2000) 

Trust  model  incorporates  four  dimensions:  competence,  predictability, 
reliability,  and  faith  in  a  system  or  machine. 

Mayer  et  al.  (1995)4* 

“[Wjillingness  of  a  party  to  be  vulnerable  to  the  actions  of  another  party 
based  on  the  expectation  that  the  other  will  perform  a  particular  action 
important  to  the  trustor,  irrespective  of  the  ability  to  monitor  or  control 
that  party”  (p.  712). 

Note:  References  marked  with  a  ❖  represent  definitions  that  are  originally  from  another  domain  of  trust  but  are  referenced  or 
applied  to  automation. 
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Definition  Citation 

Definition 

Merriam-Webster  Online  (2013) 

1)  a:  assured  reliance  on  the  character,  ability,  strength,  or  truth  of  someone 
or  something  b:  one  in  which  confidence  is  placed 

2)  a:  dependence  on  something  future  or  contingent :  hope  b:  reliance  on 
future  payment  for  property  (as  merchandise)  delivered  :  credit  <bought 
furniture  on  trust> 

3)  a:  a  property  interest  held  by  one  person  for  the  benefit  of  another  b:  a 
combination  of  firms  or  corporations  formed  by  a  legal  agreement, 
especially  :  one  that  reduces  or  threatens  to  reduce  competition 

4)  archaic:  trustworthiness 

5)  a:  (1)  a  charge  or  duty  imposed  in  faith  or  confidence  or  as  a  condition  of 
some  relationship  (2)  something  committed  or  entrusted  to  one  to  be  used 
or  cared  for  in  the  interest  of  another  b:  responsible  charge  or  office  c:  care, 
custody  <the  child  committed  to  her  trust> 

—  in  trust:  in  the  care  or  possession  of  a  trustee 

Merrit  and  Ilgen  (2008) 

“Dispositional  trust  reflects  trust  in  other  persons  (or  machines)  upon 
initially  encountering  them,  even  if  no  interaction  has  yet  taken  place”  (p. 
195). 

Merrit  and  Ilgen  (2008) 

“. . .  history-based  trust  is  founded  on  interactions  between  the  person  and 
another  person  or  machine”  (p.  195). 

Merrit  and  Ilgen  (2008) 

“. . .  the  stable,  trait-like  tendency  to  trust  or  not  trust  others,  which  we  call 
propensity  to  trust”  (p.  195). 

Moorman  et  al.  (1993) 

“Trust  is  defined  as  a  willingness  to  rely  on  an  exchange  partner  in  whom 
one  has  confidence  (Moorman,  Zaltman,  and  Deshpande,  1992)”  (p.  82). 

Moray  and  Inagaki  (1999) 

“. . .  an  attitude  which  includes  the  belief  that  the  collaborator  will  perform 
as  expected,  and  can,  within  the  limits  of  its  designers’  intentions,  be  relied 
on  to  achieve  the  design  goals”  (p.  204). 

Muir  (1989) 

The  dynamic  nature  of  trust  consist  of  three  dimensions:  predictability, 
dependability,  and  faith 

Muir  (1994) 

“. .  .an  intervening  variable  that  mediates  between  the  automated  system  and 
the  user’s  choice  of  a  control  strategy  in  his/her  interaction  with  the  system” 

Parasuraman  and  Miller  (2004) 

“In  practice,  user  trust  should  be  calibrated  to  the  system  and  context  of  its 
use,  users  should  have  accurate  beliefs  about  the  reliability  of  automation 
(and  about  their  own  capabilities)”  (p.  52). 

Rajaonah  et  al.  (2006) 

“We  define  trust  as  a  psychological  state  resulting  from  knowledge,  beliefs, 
and  assessments  (Castelffanche  &  Falcone,  2000)  related  to  the  decision 
making  situation,  that  creates  confident  expectations  (Corritore,  Kracher, 
and  Wiedenbeck,  2003)  for  human  machine  system  performance  and  guides 
operator  reliance  on  automation  (Lee  &  See,  2004)”  (p.  102). 

Rempel  et  al.  (1985)4* 

“. . .  A  generalized  expectation  related  to  the  subjective  probability  an 
individual  assigns  to  the  occurrence  of  some  set  of  future  events”  (p.  96). 

Note:  References  marked  with  a  ❖  represent  definitions  that  are  originally  from  another  domain  of  trust  but  are  referenced  or 
applied  to  automation. 
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Definition  Citation 

Definition 

Rempel  and  Holmes  (1986)4* 

“In  a  general  sense,  trust  is  the  degree  of  confidence  you  feel  when  you 
think  about  a  relationship”  (p.  28). 

Ross  (2008) 

“To  measure  objective  trust  of  an  automated  system,  reliance  was  analyzed. 
Reliance  was  measured  as  the  combined  total  of  the  times  the  participant 
agreed  with  the  aid”  (p.  38). 

Rotter  (1967)*> 

“A  generalized  expectancy  held  by  an  individual  that  the  word,  promise, 
oral  or  written  statement  of  another  individual  or  group  can  be  relied  upon” 
(p.  653). 

Scanzoni  (1979)<» 

An  actor’s  willingness  to  arrange  and  repose  his  or  her  activities  on  [an] 

Other  because  of  confidence  that  [the]  Other  will  provide  expected 
gratifications. 

Singh  etal.  (1991) 

Complacency  is  a  set  of  attitudes  involving  uncritical  trust  towards,  over 
reliance  on,  excessive  confidence  in,  and  other  related  attitudes  towards 
automation  technology. 

Swadesh  and  Kasouf  (1995) 

“Trust  is  related  to  a  number  of  characteristics  of  the  researcher:  sincerity, 
integrity,  tactfulness,  timeliness  and  ability  to  keep  information 
confidential”  (p.  199). 

Note:  References  marked  with  a  ❖  represent  definitions  that  are  originally  from  another  domain  of  trust  but  are  referenced  or 
applied  to  automation. 
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